Vision-Language-Action models (VLAs) are becoming increasingly capable across diverse robotic tasks. However, their real-world deployment remains slow and inefficient: demonstration videos are often sped up by 5-10x to appear smooth, with noticeable action stalls and delayed reactions to environmental changes. Asynchronous inference offers a promising solution to achieve continuous and low-latency control by enabling robots to execute actions and perform inference simultaneously. However, because the robot and environment continue to evolve during inference, a temporal misalignment arises between the prediction and execution intervals. This leads to significant action instability, while existing methods either degrade accuracy or introduce runtime overhead to mitigate it. We propose VLASH, a general asynchronous inference framework for VLAs that delivers smooth, accurate, and fast reaction control without additional overhead or architectural changes. VLASH estimates the future execution-time state by rolling the robot state forward with the previously generated action chunk, thereby bridging the gap between prediction and execution. Experiments show that VLASH achieves up to 2.03x speedup and reduces reaction latency by up to 17.4x compared to synchronous inference while fully preserving the original accuracy. Moreover, it empowers VLAs to handle fast-reaction, high-precision tasks such as playing ping-pong and playing whack-a-mole, where traditional synchronous inference fails. Code is available at https://github.com/mit-han-lab/vlash

The Universal Robots RTDE communication interface is well-known in literature and it was used in several works. In [5] and [6] RTDE was adopted to control UR cobots. In [7], [8], and [9], the RTDE interface was used only for data acquisition. To facilitate the development of external applications for UR cobots, various higher-level software interfaces and drivers have been proposed based on RTDE. In addition to the official software interface by Universal Robots (ur_client_li-brary), a few alternatives have been developed by third-parties. One of these software interfaces is ur_rtde [4] by SDU Robotics, which was used in this work. Another similar interface is python-urx [10], which is a Python interface for tasks that do not require high control frequency.

For many complex tasks, multi-finger robot hands are poised to revolutionize how we interact with the world, but reliably grasping objects remains a significant challenge. We focus on the problem of synthesizing grasps for multi-finger robot hands that, given a target object's geometry and pose, computes a hand configuration. Existing approaches often struggle to produce reliable grasps that sufficiently constrain object motion, leading to instability under disturbances and failed grasps. A key reason is that during grasp generation, they typically focus on resisting a single wrench, while ignoring the object's potential for adversarial movements, such as escaping. We propose a new grasp-synthesis approach that explicitly captures and leverages the adversarial object motion in grasp generation by formulating the problem as a two-player game. One player controls the robot to generate feasible grasp configurations, while the other adversarially controls the object to seek motions that attempt to escape from the grasp. Simulation experiments on various robot platforms and target objects show that our approach achieves a success rate of 75.78%, up to 19.61% higher than the state-of-the-art baseline. The two-player game mechanism improves the grasping success rate by 27.40% over the method without the game formulation. Our approach requires only 0.28-1.04 seconds on average to generate a grasp configuration, depending on the robot platform, making it suitable for real-world deployment. In real-world experiments, our approach achieves an average success rate of 85.0% on ShadowHand and 87.5% on LeapHand, which confirms its feasibility and effectiveness in real robot setups.

Abstract-- A key challenge in robot manipulation lies in developing policy models with consistent spatial understanding--the ability to reason about 3D geometry, object relations, and robot state. Existing mainstream models take 2D images as input, without performing explicit 3D modeling, and thus lack spatial understanding capabilities as well as 3D and embodiment generalization. T o address this, we propose SEM (Spatial Enhanced Manipulation), a diffusion-based policy framework that constructs a unified spatial representation by projecting multi-view image features and joint-centric robot states into a unified 3D space. This spatially aligned representation provides a consistent feature space for the diffusion policy to condition on during action generation. Extensive experiments demonstrate that SEM significantly improves spatial understanding, leading to robust and generalizable manipulation across diverse tasks that outperform existing baselines.

Simulation plays a crucial role in various robotics applications, such as policy learning [1, 2, 3, 4, 5, 6, 7], safe and scalable robotic control evaluation [8, 9, 10, 11], and computational optimization of robot designs [12, 13, 14]. Recently, neural robotics simulators have emerged as a promising alternative to traditional analytical simulators, as neural simulators can efficiently predict robot dynamics and learn intricate physics from real-world data. For instance, neural simulators have been leveraged to capture complex interactions challenging for analytical modeling [15, 16, 17, 18], or have served as learned world models to facilitate sample-efficient policy learning [19, 20]. However, existing neural robotics simulators typically require application-specific training, often assuming fixed environments [20, 21] or simultaneous training alongside control policies [22, 23]. These limitations primarily stem from their end-to-end frameworks with inadequate representations of the global simulation state, i.e., neural models often substitute the entire classical simulator and directly map robot state and control actions ( e.g., target joint positions, target link orientations) to the robot's next state. Without encoding the environment in the state representation, the learned simulators have to implicitly memorize the task and environment details. Additionally, utilizing controller actions as input causes the simulators to overfit to particular low-level controllers used during training. Consequently, unlike classical simulators, these neural simulators often fail to generalize to novel state distributions (induced by new tasks), unseen environment setups, and customized controllers ( e.g., novel control laws or controller gains).

--This paper proposes a semidefinite relaxation for landmark-based localization with unknown data associations in planar environments. The proposed method simultaneously solves for the optimal robot states and data associations in a globally optimal fashion. Relative position measurements to a fixed set of known landmarks are used, but the data association is unknown in that the robot does not know which landmark each measurement is generated from. The relaxation is shown to be tight in a majority of cases for moderate noise levels. The proposed algorithm is compared to local Gauss-Newton baselines initialized at the dead-reckoned trajectory, and is shown to significantly improve convergence to the problem's global optimum in simulation and experiment. STIMA TING the state of a robot from noisy and incomplete sensor data is a central task associated with autonomy. In the landmark-based localization task, the robot infers its position and orientation from measurements from landmarks with known positions. State estimation methods for localization can be split into filtering methods and batch optimization methods [1].

Seamless operation of mobile robots in challenging environments requires low-latency local motion estimation (e.g., dynamic maneuvers) and accurate global localization (e.g., wayfinding). While most existing sensor-fusion approaches are designed for specific scenarios, this work introduces a flexible open-source solution for task- and setup-agnostic multimodal sensor fusion that is distinguished by its generality and usability. Holistic Fusion formulates sensor fusion as a combined estimation problem of i) the local and global robot state and ii) a (theoretically unlimited) number of dynamic context variables, including automatic alignment of reference frames; this formulation fits countless real-world applications without any conceptual modifications. The proposed factor-graph solution enables the direct fusion of an arbitrary number of absolute, local, and landmark measurements expressed with respect to different reference frames by explicitly including them as states in the optimization and modeling their evolution as random walks. Moreover, local smoothness and consistency receive particular attention to prevent jumps in the robot state belief. HF enables low-latency and smooth online state estimation on typical robot hardware while simultaneously providing low-drift global localization at the IMU measurement rate. The efficacy of this released framework is demonstrated in five real-world scenarios on three robotic platforms, each with distinct task requirements.

Current efforts to learn scalable policies in robotic manipulation primarily fall into two categories: one focuses on "action," which involves behavior cloning from extensive collections of robotic data, while the other emphasizes "vision," enhancing model generalization by pre-training representations or generative models, also referred to as world models, using large-scale visual datasets. This paper presents an end-to-end paradigm that predicts actions using inverse dynamics models conditioned on the robot's forecasted visual states, named Predictive Inverse Dynamics Models (PIDM). By closing the loop between vision and action, the end-to-end PIDM can be a better scalable action learner. In practice, we use Transformers to process both visual states and actions, naming the model Seer. It is initially pre-trained on large-scale robotic datasets, such as DROID, and can be adapted to realworld scenarios with a little fine-tuning data. Thanks to large-scale, end-to-end training and the synergy between vision and action, Seer significantly outperforms previous methods across both simulation and real-world experiments. It achieves improvements of 13% on the LIBERO-LONG benchmark, 21% on CALVIN ABC-D, and 43% in real-world tasks. Notably, Seer sets a new state-of-the-art on CALVIN ABC-D benchmark, achieving an average length of 4.28, and exhibits superior generalization for novel objects, lighting conditions, and environments under high-intensity disturbances on real-world scenarios. Code and models are publicly available at https://github.com/OpenRobotLab/Seer/.

Environments with large terrain height variations present great challenges for legged robot locomotion. Drawing inspiration from fire ants' collective assembly behavior, we study strategies that can enable two ``connectable'' robots to collectively navigate over bumpy terrains with height variations larger than robot leg length. Each robot was designed to be extremely simple, with a cubical body and one rotary motor actuating four vertical peg legs that move in pairs. Two or more robots could physically connect to one another to enhance collective mobility. We performed locomotion experiments with a two-robot group, across an obstacle field filled with uniformly-distributed semi-spherical ``boulders''. Experimentally-measured robot speed suggested that the connection length between the robots has a significant effect on collective mobility: connection length C in [0.86, 0.9] robot unit body length (UBL) were able to produce sustainable movements across the obstacle field, whereas connection length C in [0.63, 0.84] and [0.92, 1.1] UBL resulted in low traversability. An energy landscape based model revealed the underlying mechanism of how connection length modulated collective mobility through the system's potential energy landscape, and informed adaptation strategies for the two-robot system to adapt their connection length for traversing obstacle fields with varying spatial frequencies. Our results demonstrated that by varying the connection configuration between the robots, the two-robot system could leverage mechanical intelligence to better utilize obstacle interaction forces and produce improved locomotion. Going forward, we envision that generalized principles of robot-environment coupling can inform design and control strategies for a large group of small robots to achieve ant-like collective environment negotiation.

As humanoid robots transition from labs to real-world environments, it is essential to democratize robot control for non-expert users. Recent human-robot imitation algorithms focus on following a reference human motion with high precision, but they are susceptible to the quality of the reference motion and require the human operator to simplify its movements to match the robot's capabilities. Instead, we consider that the robot should understand and adapt the reference motion to its own abilities, facilitating the operator's task. For that, we introduce a deep-learning model that anticipates the robot's performance when imitating a given reference. Then, our system can generate multiple references given a high-level task command, assign a score to each of them, and select the best reference to achieve the desired robot behavior. Our Self-AWare model (SAW) ranks potential robot behaviors based on various criteria, such as fall likelihood, adherence to the reference motion, and smoothness. We integrate advanced motion generation, robot control, and SAW in one unique system, ensuring optimal robot behavior for any task command. For instance, SAW can anticipate falls with 99.29% accuracy. For more information check our project page: https://evm7.github.io/Self-AWare